Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing responsibilities #103

Merged
merged 10 commits into from
Nov 21, 2023
Merged

Conversation

aufi
Copy link
Member

@aufi aufi commented Mar 9, 2023

As a follow-up on #98 "Add CI testing overview" and an exploration on devs&QE cooperation on our recent migration projects, this enhacement attempts to specify expected responsibilities and Konveyor testing work coordination.

| **integration** Hub | Hub developers | REST API coverage tests, applications import, [more](https://github.com/konveyor/tackle2-hub/discussions/241) | https://github.com/konveyor/tackle2-hub/... | PR&push (+time anyway?) |
| **integration** addon-windup | Addon developers | Bash-scripted windup analysis | https://github.com/konveyor/tackle2-addon-windup/blob/main/hack/test-e2e.sh | PR&push (+time anyway?) |
||||||
| **E2E** API | QE&Developers | Golang API test suite (WIP), focusing on sanity checks | https://github.com/konveyor/go-konveyor-tests | time-based schedule |
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From discussion with @jwmatthews - it might make sense to require a developer responsible for a feature to deliver E2E "sunny day" test to strenghten user focus and E2E responsibility for the feature. (QE should use the provided test as a startingpoint for their scenarios on the feature tests).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'd like to make this a requirement going forward. The idea is we want developers contributing a feature to also contribute an automated means of showing the end-to-end scenario is working as expected.


### Execution and project status

Konveyor CI repository is https://github.com/konveyor/ci
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The konveyor/ci repo might not be needed if we put CI/test execution workflow badges to konveyor/tackle2-operator README. (and testing guidelines e.g. to Hub's test README or tackle2-addon)


| Kind of test | Responsible | Description | Tests code | Trigger (min.required) |
|---|---|---|---|---|
| **integration** Hub | Hub developers | REST API coverage tests, applications import, [more](https://github.com/konveyor/tackle2-hub/discussions/241) | https://github.com/konveyor/tackle2-hub/... | PR&push (+time anyway?) |
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Draft PR with Hub API tests: konveyor/tackle2-hub#268

@sshveta
Copy link

sshveta commented Mar 27, 2023

Looks Good @aufi .
Assuming that these tests will be run on downstream builds , we also have scripts to install MTA on a cluster and check status of all pods once deployed.
This can also help in checking sanity of components/builds .
So a build can be considered Green if MTA deployed successfully, all pods came up and then some sanity/API or UI tests passed .

@jwmatthews
Copy link
Member

@aufi @fabianvf @dymurray please take a look and see if this is good to review/ack and merge

@aufi
Copy link
Member Author

aufi commented Nov 2, 2023

I'm going to do a minor update based on CI work from last months to make it up-to-date and ready for review/merge.

@aufi
Copy link
Member Author

aufi commented Nov 14, 2023

Updated enhancement with few CI notes and removing not relevant parts. Ready for review.

@aufi aufi requested a review from jwmatthews November 14, 2023 10:31
Signed-off-by: Marek Aufart <[email protected]>
@aufi aufi merged commit 1f45850 into konveyor:master Nov 21, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants